Story: 2003120
Task: 23239

This removes the deprecated Gnocchi dispatcher and replaces it by its
equivalent publisher.

Change-Id: Ie44baf20ccb8de5794f5f0c3d4717f7e56afa63b
(cherry picked from commit 228517e)
original commit 83ffaff
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

ujson is faster than jsonutils, and we do not need any fancy feature jsonutils
might offer.

This also has the benefit of removing a big dependency on Ceilometer.

Change-Id: I24bf08d0fa6ccc34beef0a0c34a47bf2fa266e3e
(cherry picked from commit 886dc38)
original commit 7b959b3
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

don't use lambda to filter. this is 25% faster but really negligible.

Change-Id: Ib76470fc3a5cdfc79e497d568147b66c08e6dba1
(cherry picked from commit 992efae)
original commit 8e3ba4d
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

ujson has not had any active maintenance for the last 12 months;
switch to using json module instead.

Change-Id: I39027b534e94b3f877d881647a7c843183f60f92
Closes-Bug: 1737989
(cherry picked from commit 5ebc250)
original commit 05050e4
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

we shouldn't waste our time looping through resources and searching
metrics to find a match. just build hashmap and find the resource
based on metric.

this removes fnmatch functionality because we don't match on
wildcards and the code itself actually requires metric names to be
explicit to create_metrics

Change-Id: I2398247270217759c876ab5a9b60038dad79a9d3
(cherry picked from commit e516e82)
original commit 8099a94
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

we shouldn't be wasting time doing the same thing over and over
in loop

Change-Id: Iaa5aeb5b91004945a24fab105c8b4884918c1644
(cherry picked from commit 7275dd5)
original commit 40317d7
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

don't bother with a running count to build stats

Change-Id: I6c40e49046115512d5858a6e1348629e10d6a42f

Conflicts:
	ceilometer/publisher/gnocchi.py

(cherry picked from commit c473be4)
original commit 02e8ac2
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

we run through samples and format samples and resources to what
gnocchi expects. this simplifies that process a little:
- don't needlessly sort and group by metric name
- build a full resource model iff resource needs to be created

Change-Id: I2ac25b3b0978eed664c500e645bae2d1b4ae6781
(cherry picked from commit e58fe24)
original commit 5b216fa
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

Once I delete resource manually by gnocchi client,
the resource won't be created by the ceilometer until 600s passed，
because the resource has cached in the memcached.

Change-Id: I601e39b4c4782276daba3f9d1b7be92f09efabfc
Closes-Bug: #1718570
(cherry picked from commit 7838fe0)
original commit 02a012660aebe171627de0922783a9da8be8fddf
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

only the attributes of a resource may change so there's no reason
to capture the entire resource in cache. this means we don't need
to cache if no attributes.

this also changes code so we only compute attribute_hash once.

Change-Id: I1363e0b0dc6e0957a7e933536b67737bd3496cbd
(cherry picked from commit 64f36fc)
original commit 236cfbeea6ab40c94af5f64c5843ce4b62f4d677
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

This creates two archive policies "ceilometer-low" and
"ceilometer-low-rate" in Gnocchi. And use them to create metrics
in Gnocchi.

The archive policies settings can now be configured in
gnocchi_resources.yaml.

This requires upcoming release of coming
(gnocchixyz/gnocchi#655)

Change-Id: Ib804ef4652af9d40338ad970690f6e92f12ed636

Conflicts:
	ceilometer/publisher/data/gnocchi_resources.yaml
	doc/source/admin/telemetry-measurements.rst

(cherry picked from commit bb4692d)
original commit b96b16928ca84bd1c4025c3f996a65e447a8920f
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

We can't create archive policies in publisher init because if it fail
because Gnocchi is not yet ready the publisher fail to load. Ceilometer
will work, but samples will go to nowhere.

This change creates/checks archive policies when we publish a sample for
the very first time. If that fail because Gnocchi is not ready, this
will just retry next time Ceilometer will publish samples.

Closes-Bug: #1752420

Change-Id: Ib6b4da54592ad99a4e6561a73473b6c7ec73a21f
(cherry picked from commit 717e968)
original commit 14906f8672282b21467d705b316f0b8ec88c6bee
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

event is now an object.

Change-Id: I22fbb797dc9061240068ff2b4e0b3a2d76463248
(cherry picked from commit 27064b6)
original commit 0f14afe
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407

Ceilometer creates metrics that will never get measures (service not
polled, notification not configured).

This is because we create all metrics during resource creation. That was
done sometimes ago because Gnocchi was not provide metrics creating
during batching process.

But since a while now, we also create metrics (if missing) during the
batch processing.

This change removes the creation of metrics during resource creation.
All metrics with measures got created during the batch call.

This stops to have metric without measures in Gnocchi.

Change-Id: I3f3b805722b11029aefdb76bae93413cb32ecff6
(cherry picked from commit 10cd6ba)
original commit 826ba35c6eb9900bb0a557f6e4f06f7d1b9bd394
StoryBoard:
“Various backports to core openstack components to pick up fixes”
https://storyboard.openstack.org/#!/story/2003407